Modern computing and display technologies have facilitated the development of “mixed reality” (MR) systems for so called “virtual reality” (VR) or “augmented reality” (AR) experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A VR scenario typically involves presentation of digital or virtual image information without transparency to actual real-world visual input. An AR scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the real world around the user (i.e., transparency to real-world visual input). Accordingly, AR scenarios involve presentation of digital or virtual image information with transparency to the real-world visual input.
MR systems typically generate and display color data, which increases the realism of MR scenarios. Many of these MR systems display color data by sequentially projecting sub-images in different (e.g., primary) colors or “fields” (e.g., Red, Green, and Blue) corresponding to a color image in rapid succession. Projecting color sub-images at sufficiently high rates (e.g., 60 Hz, 120 Hz, etc.) may deliver a smooth color MR scenarios in a user's mind.
For example, referring to FIG. 1, an augmented reality scene 4 is depicted wherein a user of an AR/MR technology sees a real-world park-like setting 6 featuring people, trees, buildings in the background, and a concrete platform 8. In addition to these items, the end user of the AR/MR technology also perceives that he “sees” a robot statue 10 standing upon the real-world platform 8, and a cartoon-like avatar character 12 flying by which seems to be a personification of a bumble bee, even though these elements 10, 12 do not exist in the real world. As it turns out, the human visual perception system is very complex, and producing a VR, AR, and/or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Some VR, AR, and/or MR systems employ head-worn displays (or helmet-mounted displays, or smart glasses) that are at least loosely coupled to a user's head, and thus move when the end user's head moves. If the end user's head motions are detected by the display subsystem, the data being displayed can be updated to take the change in head pose (i.e., the orientation and/or location of user's head) into account. Head-worn displays that enable AR/MR (i.e., the concurrent viewing of real and virtual objects) can have several different types of configurations. In one such configuration, often referred to as a “video see-through” display, a camera captures elements of a real scene, a computing system superimposes virtual elements onto the captured real scene, and a non-transparent display presents the composite image to the eyes. Another configuration is often referred to as an “optical see-through” display, in which the end user can see through transparent (or semi-transparent) elements in the display subsystem to view directly the light from real objects in the environment. The transparent element, often referred to as a “combiner,” superimposes light from the display over the end user's view of the real world.
Some head-worn VR/AR/MR systems employ a display screen in the field of view of the end user and an image projection assembly that projects images onto the display screen. As one example, the image projection assembly may take the form of an optical fiber scan-based image projection assembly, and the display screen may take the form of a optical waveguide-based display into which scanned and collimated light beams from the image projection assembly are injected via an in-coupling (IC) element, which the exit the surface of the optical waveguide-based display towards the user's eyes, thereby producing, e.g., images at single optical viewing distance closer than infinity (e.g., arm's length), images at multiple, discrete optical viewing distances or focal planes, and/or image layers stacked at multiple viewing distances or focal planes to represent volumetric 3D objects.
In a head-worn VR/AR/MR system, it is important that the entrance pupil of the user's eye (i.e., the image of the anatomical pupil as seen through the cornea) be aligned with and be of a similar size to the exit pupil of the display screen (i.e., the width of the cone of light that is available to the eye of the user) in order to properly couple the instrument to the eye (in the case of a monocular arrangement) or eyes (in the case of a binocular arrangement) of the user, given a fixed eye relief (i.e., the distance from the last surface of the display screen and the user's eye or eyes). An exit pupil of the display screen that is smaller than the entrance pupil of the user' eye will often result in a vignette or clipped image, whereas an exit pupil of the display screen that is larger than the entrance pupil of the user's eye wastes some light, but allows for movement of the eye without vignetting or clipping of the image.
In order to increase the wearability and comfort of a head-worn VR/AR/MR system, it is desirable to miniaturize the image source, and in some cases, the image projection assembly, as much as possible. Such an image projection assembly will, without intervention, result in an exit pupil that is much smaller than the entrance pupil of some eyes, assuming a reasonable eye relief between the eye and the display screen. As such, optics are incorporated into the display subsystem to effectively expand the exit pupil of the display screen to match the entrance pupil of the user's eye. That is, the exit pupil of the display screen should create an “eye box” that is slightly larger (e.g., 10 mm) than the entrance pupil of the user's eye (e.g., 5-7 mm) to allow movement of the eye within that eye box to maintain a full view of the image presented by the display screen.
Besides matching the exit pupil of the display screen with the entrance pupil of the user's eye(s), it is desirable to maximize the angular resolution, minimize the depth of field, and maximize the density of the wavefront density of the display screen in a VR/AR/MR system. Maximizing the angular resolution results in a clearer and more vivid virtual image, maximizing the wavefront density alleviates image artifacts (such as the “screen door” effect (grid-like pattern and non-uniformity), and minimizing the depth of the field allows the user to more easily accommodate to virtual content on which the user is currently focused. That is, the smaller the depth of field, the easier it is for an eye to accommodate to the virtual content, providing for a more natural visual real-world experience, whereas the greater the depth of field, the more difficult it is for the eye to accommodate to the virtual content, resulting in a less natural, and perhaps a nauseating, visual experience.
There, thus, remains a need to provide a display screen of a VR/AR/MR system that is capable of producing a highly-saturated light beamlet array exit pupil that matches the entrance pupil of the user's eye(s), without diminishing the wearability of the VR/AR/MR system.
The visualization center of the brain gains valuable perception information from the motion of both eyes and components thereof relative to each other. Vergence movements (i.e., rolling movements of the pupils toward or away from each other to converge the lines of sight of the eyes to fixate upon an object) of the two eyes relative to each other are closely associated with focusing (or “accommodation”) of the lenses of the eyes. Under normal conditions, changing the focus of the lenses of the eyes, or accommodating the eyes, to focus upon an object at a different distance will automatically cause a matching change in vergence to the same distance, under a relationship known as the “accommodation-vergence reflex.” Likewise, a change in vergence will trigger a matching change in accommodation, under normal conditions. Working against this reflex, as do most conventional stereoscopic VR/AR/MR configurations, is known to produce eye fatigue, headaches, or other forms of discomfort in users.
Stereoscopic wearable glasses generally feature two displays for the left and right eyes that are configured to display images with slightly different element presentation such that a three-dimensional perspective is perceived by the human visual system. Such configurations have been found to be uncomfortable for many users due to a mismatch between vergence and accommodation (“vergence-accommodation conflict”) which must be overcome to perceive the images in three dimensions. Indeed, some VR/AR/MR users are not able to tolerate stereoscopic configurations. Accordingly, most conventional VR/AR/MR systems are not optimally suited for presenting a rich, binocular, three-dimensional experience/scenario in a manner that will be comfortable and maximally useful to the user, in part because prior systems fail to address some of the fundamental aspects of the human perception system, including the vergence-accommodation conflict.
One possible approach to address these problems (including the vergence-accommodation conflict) is to project images at multiple depth planes. To implement this type of system, one approach is to use a plurality of light guiding optical elements to direct light at the eyes of a user such that the light appears to originate from multiple depth planes. The light guiding optical elements are designed to in-couple virtual light corresponding to digital or virtual objects and propagate it by total internal reflection (“TIR”), then to out-couple the virtual light to display the digital or virtual objects to the user's eyes. In AR/MR systems, the light guiding optical elements are also designed to be transparent to light from (e.g., reflecting off of) actual real-world objects. Therefore, portions of the light guiding optical elements are designed to reflect virtual light for propagation via TIR while being transparent to real-world light from real-world objects.
Various optical systems generate images at various depths for displaying VR/AR/MR scenarios. Some such optical systems are described in U.S. Utility patent application Ser. No. 14/555,585, the contents of which have been previously incorporated by reference. Some VR/AR/MR systems employ wearable display devices (e.g., head-worn displays, helmet-mounted displays, or smart glasses) that are at least loosely coupled to a user's head, and thus move when the user's head moves.
Some three-dimensional (“3-D”) optical systems, such as those in VR/AR/MR systems, optically render virtual objects. Objects are “virtual” in that they are not real physical objects located in respective positions in 3-D space. Instead, virtual objects only exist in the brains (e.g., the optical centers) of viewers and/or listeners when stimulated by light beams directed to the eyes of audience members.
VR/AR/MR systems must also be capable of displaying virtual digital content at various perceived positions and distances relative to the user. The design of VR/AR/MR systems presents numerous other challenges, including the speed of the system in delivering virtual digital content, quality of virtual digital content, eye relief of the user (addressing the vergence-accommodation conflict), size and portability of the system, and other system and optical challenges.
Further, VR/AR/MR systems must be capable of displaying virtual digital content in sharp focus to generate photo-realistic imagery required for a believable, immersive, enjoyable VR/AR/MR experience/scenario. The lens of an eye must change shape (i.e., accommodate) to bring images or portions thereof into better focus.
Size restrictions of head-worn displays also result in image resolution limitations. Head-worn VR/AR/MR display systems, such as those described in U.S. Utility patent application Ser. No. 14/555,585, the contents of which have been previously incorporated by reference, display images to users with light beams transmitted by TIR through light guiding optical elements which conserve light beam angles. Light beam diameters remain essentially the same through light guiding optical elements. Size limitations of head-worn displays limited the size of various optical components (e.g., light sources, light guiding optical elements, lenses, etc.), which limits the diameters of light beams generated by the head-worn displays. These light beam diameter limitations result in resolution and FOV limitations described above.
The systems and methods described herein are configured to address these challenges.